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PREFACE 


This  report  documents  the  results  of  AFCCC  (formerly  USAFETAC)  Project  9411010,  accomplished  in  response 
to  a  request  from  the  42nd  OSS/OSFW  at  Maxwell  AFB,  Ala.  The  requester  asked  AFCCC  to  correlate  various 
thunderstorm  indices  to  the  occurrence/non-occurrence  of  thunderstorms  in  order  to  improve  their  forecasting 
capability.  The  request  was  subsequently  modified  to  include  regression  equations  that  provide  probability  forecast 
guidance  and  80th  percentile  threshold  values  of  each  index.  Thunderstorm  prediction  continues  to  be  an  important 
and  challenging  aspect  of  operational  aviation  forecasting  in  the  military.  Most  of  research  and  investigation 
has  been  devoted  to  forecasting  the  intensity  of  thunderstorms — not  whether  or  not  thunderstorms  will  occur  at 
a  given  location.  This  has  left  the  forecaster  with  the  tools  to  forecast  how  severe  the  thunderstorm  would  be  if 
it  were  to  happen  and  nothing  to  guide  them  on  how  to  forecast  whether  or  not  thunderstorms  will  occur.  This 
study  was  an  attempt  to  provide  a  tool  to  this  end,  using  standard  thunderstorm  indices. 


iv 


CONTENTS 


CHAPTER  1.  INTRODUCTION . 

1.1  Purpose . 

1.2  Components  of  the  Study . 

CHAPTER  2.  DISCRIMINATE  ANALYSIS 

2.1  Introduction . 

2.2  Methodology . 

2.3  Predictor  Variables . 

2.4  Discriminate  Function . 

2.5  Classification  Summary . 


CHAPTER  3.  VERIFICATION . 

3.1  Verification  Procedure 

3.2  Verification  Results.... 

3.3  Skill  Scores . 


CHAPTER  4.  SUMMARY,  CONCLUSIONS,  AND  RECOMMENDATIONS 

4.1  Summary . . . 

4.2  Conclusion . 

4.3  Recommendations . 


GLOSSARY . 

ACRONABs . 

BIBLIOGRAPHY 


v 


TABLES 


Table  1.  Thunderstorm  indices  binning  scheme  used  in  the  discriminate  analysis . 3 

Table  2.  Classification  table  for  the  unmodified  Centerville,  Ala.  upper-air  data  using 

binned  indices  for  the  calibration  data . 5 

Table  3.  Classification  table  for  the  unmodified  Centerville,  Ala.,  upper-air  data  using 

unbinned  indices  for  the  calibration  data . 5 

Table  4.  Classification  table  for  the  modified  Centerville,  Ala.,  upper-air  data  using 

binned  indices  for  the  calibration  data . 5 

Table  5.  Classification  table  for  the  modified  Centerville,  Ala.,  upper-air  data  using 

unbinned  indices  for  the  calibration  data . 5 

Table  6.  Statistical  summary  for  the  calibration  data  showing  percent  correct,  percent 

incorrect,  percent  missed  and  false  alarm  rate . 5 

Table  7.  Classification  table  for  binned  unmodified  upper-air  data  (50  percent) . 7 

Table  8.  Classification  table  for  unbinned  unmodified  upper-air  data  (50  percent) . 7 

Table  9.  Classification  table  for  binned  modified  upper-air  data  (50  percent) . 7 

Table  10.  Classification  table  for  unbinned  modified  upper-air  data  (50  percent) . 7 

Table  1 1 .  Classification  table  for  binned  unmodified  upper-air  data  (55  percent) . 8 

Table  12.  Classification  table  for  unbinned  unmodified  upper-air  data  (55  percent) . 8 

Table  13.  Classification  table  for  binned  modified  upper-air  data  (55  percent) . 8 

Table  14.  Classification  table  for  unbinned  modified  upper-air  data  (55  percent) . 8 

Table  15.  Classification  table  for  binned  unmodified  upper-air  data  (60  percent) . 8 

Table  16.  Classification  table  for  unbinned  unmodified  upper-air  data  (60  percent) . 8 

Table  17.  Classification  table  for  binned  modified  upper-air  data  (60  percent) . 8 

Table  18.  Classification  table  for  unbinned  modified  upper-air  data  (60  percent) . 8 

Table  19.  Classification  table  for  binned  unmodified  upper-air  data  (65  percent) . 9 

Table  20.  Classification  table  for  unbinned  unmodified  upper-air  data  (65  percent) . 9 

Table  21 .  Classification  table  for  binned  modified  upper-air  data  (65  percent) . 9 

Table  22.  Classification  table  for  unbinned  modified  upper-air  data  (65  percent) . 9 


vi 


9 


Table  23. 
Table  24. 
Table  25. 
Table  26. 
Table  27. 
Table  28. 
Table  29. 
Table  30. 
Table  31. 
Table  32. 


Classification  table  for  binned  unmodified  upper-air  data  (70  percent) . 

Classification  table  for  unbinned  unmodified  upper-air  data  (70  percent) . 

Classification  table  for  binned  modified  upper-air  data  (70  percent) . 

Classification  table  for  unbinned  modified  upper-air  data  (70  percent) . 

Classification  table  for  binned  unmodified  upper-air  data  (75  percent) . 

Classification  table  for  unbinned  unmodified  upper-air  data  (75  percent) . 

Classification  table  for  binned  modified  upper-air  data  (75  percent) . 

Classification  table  for  unbinned  modified  upper-air  data  (75  percent) . 

Sample  contingency  table . 

Summary  of  skill  table  showing  the  forecast  threshold,  percent  correct,  percent 
incorrect,  probability  of  detection,  percent  missed,  false  alarm  rate,  Heidke  skill 
score,  and  Hanssen  and  Kuipers  Discriminate . 


9 

9 

9 

10 
10 
10 
10 

.  10 

.11 


vii 


MAXWELL  AFB  THUNDERSTORM  STUDY 

Chapter  1 

INTRODUCTION 


1.1  Background.  The  Maxwell  AFB  weather  unit 
provides  weather  support  for  a  wide  variety  of 
customers.  From  May  to  October,  air  mass 
thunderstorms  have  a  negative  impact  on  these 
customer's  operations.  Since  accurate  thunderstorm 
forecasting  is  crucial  to  all  activities  at  Maxwell  AFB, 
the  local  weather  unit  tasked  AFCCC  to  correlate 
standard  thunderstorm  and  stability  indices  to  the 
occurrence/non-occurrence  of  air  mass  thunderstorms. 
The  indices  were  calculated  first  using  the  1200Z 
Centerville,  Ala.,  sounding,  and  then  using  the  same 
sounding  modified  at  the  surface  by  the  1800Z 
Maxwell  AFB  observation.  Additionally,  the  80th 
percentile  for  these  indices  was  calculated. 

Initial  investigation  into  the  correlation  quickly 
showed  very  low  correlation  between  the  indices  and 
the  occurrence/non-occurrence  of  thunderstorms. 
This  is  primarily  due  to  the  fact  that  most  of  the  indices 
were  developed  to  determine  the  severity  of 
thunderstorms,  rather  than  to  determine  whether  or 
not  they  will  occur.  After  consulting  with  the 
requester,  the  original  request  for  correlation  was 
changed  to  regression  analysis  to  provide  probability 
of  occurrence  information. 

1.2  Components  of  the  Study. 

1.2.1  Data  Extraction.  The  data  used  in  this  study 
consisted  of  the  1200Z  upper-air  observations  from 
Centerville,  Ala.,  (Block  Station  Number  722290),  and 
surface  observations  from  Maxwell  AFB  (Block 
Station  Number  722265)  and  Montgomery,  Ala., 
(Block  Station  Number  722260).  Maxwell  AFB 
1800Z  surface  observations  were  used  to  modify  the 
Centerville  upper-air  data.  Maxwell  AFB  and 
Montgomery  surface  observations  between  1600Z  and 
0200Z  were  used  to  verify  the  occurrence/non- 
occurrence  of  thunderstorms.  The  period  of  record 
(POR)  for  the  study  was  May  through  October  for  the 
years  1982  to  1992.  The  requester  asked  that  1993  be 
used  as  the  verification  year,  but  the  POR  from  1975 
to  1981  was  used  to  ensure  a  statistically  significant 
sample  size. 


1.2.2  Sounding  Modification.  The  raw  upper-air  data 
was  quality  controlled,  preprocessed,  and  interpolated 
every  1,000  feet  from  the  surface  to  40,000  feet.  The 
interpolation  was  needed  in  order  to  ensure  the  most 
accurate  analysis  and  to  facilitate  the  use  of  a 
FORTRAN  version  of  the  Skew-T  Hodograph 
Analysis  and  Research  Program  (SHARP)  routines 
to  calculate  the  indices.  The  use  of  SHARP  code  was 
convenient  and  applicable  since  the  requester  routinely 
uses  the  MS-DOS  version  of  the  SHARP  program  to 
calculate  thunderstorm  indices.  The  1800Z  Maxwell 
surface  observation  of  temperature  and  dew  point  was 
substituted  for  the  surface  values  of  temperature  and 
dew  point  in  the  upper-air  data.  No  other 
modifications  were  made. 

1.2.3  Indices  Calculation.  Eleven  thunderstorm 
indices  were  calculated  using  the  unmodified  and 
modified  Centerville  upper-air  data.  In  both 
calculations,  the  lowest  kilometer  (approximately 
3,282  feet)  was  mixed  and  any  super-adiabatic  lapse 
rates  were  eliminated  by  changing  the  lower-level  data 
to  dry  adiabatic  from  the  surface  to  the  first  point  of 
intersection  with  the  actual  temperature  curve.  The 
indices  calculated  include:  convectively  available 
potential  energy  (CAPE),  cap  or  lid  strength,  mean 
surface  to  900  mb  relative  humidity  (MeanRH),  700- 
500mb  lapse  rate  (LAPS75),  lifted  index  (LI),  vertical 
totals  (VT),  cross  totals  (CT),  total  totals  (IT),  K- 
index  (KI),  Showalter  stability  index  (SSI),  and  severe 
weather  threat  index  (SWEAT). 

1.2.4  Discriminate  Analysis.  Discriminate  analysis 
techniques  were  used  to  determine  statistically  which, 
if  any,  of  the  indices  could  be  used  as  predictors  for 
the  occurrence/non-occurrence  of  thunderstorms.  In 
order  to  perform  the  discriminate  analysis,  the  upper- 
air  data  was  classified  into  thunderstorm  (category  1) 
and  non-thunderstorm  (category  0)  days  based  on  the 
observational  data  from  Maxwell  AFB  and 
Montgomery  for  the  same  day  between  1600Z  and 
0200Z.  Once  the  data  was  classified,  the  STEPWISE 
procedure  from  the  Statistical  Analysis  System  (S  AS) 
software  package  was  used  to  determine  the  best 
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indices  to  use  in  the  discriminate  analysis  as  predictors 
of  thunderstorm  occurrence/non-occurrence.  The 
DISCRIM  procedure  was  then  used  to  develop  a 
model,  or  discriminate  function,  that  could  be  used  to 
assign  the  probability  of  the  occurrence/non¬ 


occurrence  of  thunderstorms.  This  model  was  verified 
using  an  independent  data  set.  Finally,  the  80th 
percentile  of  each  of  the  indices  was  calculated.  This 
provides  information,  for  example,  that  80  percent  of 
the  thunderstorm  days  had  CAPE  >  3675. 


•? 
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MAXWELL  AFB  THUNDERSTORM  STUDY 

Chapter  2 

DISCRIMINATE  ANALYSIS 


2.1  Introduction.  Discriminate  analysis  classifies 
individual  observations  into  groups.  It  is  a  supervised 
classification  in  that  the  groups  where  the  observations 
end  up  are  pre-identified.  In  this  study,  the  groups  are 
defined  as  a  “Yes”  or  “No”  classification  of  the 
occurrence  of  thunderstorms. 

2.2  Methodology.  As  was  mentioned  in  Chapter  1, 
two  independent  data  sets  were  used  for  this  study. 
The  analysis  portion  used  data  from  May  through 
October,  from  the  POR  1982  to  1992.  Indices  were 
calculated  from  the  unmodified  and  modified  upper- 


air  data  and  are  treated  independently.  Some  of  the 
indices  have  large  ranges  (CAPE  can  span  three  orders 
of  magnitude);  therefore,  a  binning  technique  was 
developed  to  assist  in  the  discriminate  analysis.  Four 
discriminate  functions  were  developed.  The  data  used 
in  these  four  functions  include:  the  unbinned  indices 
from  the  unmodified  upper-air  data;  the  unbinned 
indices  from  the  modified  upper-air  data;  the  binned 
indices  from  the  unmodified  upper-air  data;  and  the 
binned  indices  from  the  modified  upper-air  data.  The 
variables  used  in  each  function  are  different.  Table  1 
shows  the  bins  used  for  each  of  the  indices. 


Table  1 .  Thunderstorm  index  binning  scheme  used  in  the  discriminate  analysis. 
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2.3  Predictor  Variables.  In  order  to  determine  which 
variables  are  the  best  predictors  to  use  in  the 
discriminate  analysis,  a  stepwise  selection  was 
employed.  This  procedure  uses  analysis  of  variance 
techniques  to  examine  individual  and  groups  of  data 
to  determine  the  best  possible  combination. 

2.3.1  Unmodified  Upper-air  Indices.  The  stepwise 
analysis  determined  seven  predictor  variables 
provided  the  best  combination  for  the  binned  indices 
and  five  predictor  variables  for  the  unbinned  indices 
for  the  unmodified  upper-air  case.  The  binned 
predictors  used  were  K-index  (KI),  CAPE,  cross  totals 


(CT),  lifted  index  (LI),  SWEAT  Index,  cap  or  lid 
strength,  and  vertical  totals  (VT).  The  unbinned 
predictors  used  were  CAPE,  surface  to  900  mb  mean 
relative  humidity  (MEANRH),  total  totals  (IT),  KI, 
and  CT.  These  predictor  variables  were  then  used  in 
the  discriminate  analysis  to  generate  the  discriminate 
function  or  regression  equation. 

2.3.2  Modified  Upper-air  Indices.  The  stepwise 
analysis  determined  four  predictor  variables  provided 
the  best  combination  for  the  binned  indices  and  four 
predictor  variables  for  the  unbinned  indices  for  the 
unmodified  upper-air  case.  The  binned  predictors  used 
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were  KI,  CAPE,  CT,  and  SWEAT  Index.  The  unbinned 
predictors  used  were  CAPE,  700-500  mb  lapse  rate 
(LAPS75),  TT,  and  KI. 

2.4  Discriminate  Function.  The  discriminate 
functions  were  computed  using  the  predictor  variables 
determined  in  the  stepwise  analysis.  The  functions 
consist  of  a  constant  and  a  coefficient  multiplied  by 
each  of  the  predictor  variables.  An  example  PROC 
DISCRIM  statement  used  to  generate  the  discriminate 
function  is  shown  below.  The  procedure  call  uses  a 
data  set  called  indices  as  input,  which  contains  either 
the  unbinned  value  or  the  binned  value  of  the  indices 
and  an  indicator,  class=trwcode,  that  indicates  if  that 
day  is  or  is  not  a  thunderstorm  day.  The  pool=yes 
option  forces  the  procedure  to  use  the  pooled 
covariance  matrix  as  a  measure  of  generalized  squared 
distance.  The  var=  statement  tells  the  procedure  which 
variables  (indices)  to  use  in  the  discriminate  function. 

proc  discrim  data=indices  pool=yes; 

class=trwcode; 

var=ki  cape  ct  sweat; 

run; 

Once  the  discriminate  function  is  defined,  Equation  1 
is  used  to  calculate  the  probability  (P)  of 
thunderstorms.  In  order  to  calculate  P,  two 
coefficients,  A  and  B,  must  be  calculated.  These 


coefficients  are  computed  for  each  separate 
discriminate  function  as  described  below. 

P  =  [e<AB>+l]-1  (1) 

2.4.1  Unmodified  Discriminate  Function 
Description.  The  two  coefficients  for  the  unmodified, 
unbinned  discriminate  function  are  calculated  using 
Equations  2  and  3.  Likewise,  the  two  coefficients  for 
the  unmodified  binned  discriminate  function  are 
calculated  using  Equations  4  and  5. 

2.4.2  Modified  Discriminate  Function  Description. 
The  two  coefficients  for  modified  unbinned 
discriminate  function  are  calculated  using  Equations 
6  and  7.  Likewise,  the  two  coefficients  for  the 
modified  binned  discriminate  function  are  calculated 
using  Equations  8  and  9. 

2.5  Classification  Summary.  The  analysis  dataset 
contained  pairs  of  surface  and  upper-air  observations. 
Tables  2  through  5  show  the  classification  summary 
for  this  “calibration”  data.  To  generate  the 
classification  summary,  the  analysis  data  set  was 
resubstituted  into  the  discriminate  function  that  was 
developed  from  the  analysis  data  set.  These 
contingency  tables  show  the  number  of  occurrences 
and  percentages  of  thunderstorms  that  were  forecast 
and  observed.  Table  6  summarizes  the  percent  correct, 


A  =  -105.39698  -  0.00321  •  CAPE  +  0.91225  •  MEANRH  +  6.58412  •  7T  -  0.55431  •  KI  -  6.63435  •  CT  (2) 
B  =  -109.09721  -  0.00284  •  CAPE  +  0.92301  •  MEANRH  +  6.66081  •  TT  -  0.49578  •  KI  -  6.76192  •  CT  (3) 


A  =  -70.19581  +  3.68631  •  KI  - 1.7549  •  CAPE  +  2.49144  •  CT  +  7.2467  •  LI  +  5.54527  •  SWEAT  - 

1.08803  •  CAP  +  5.44401  •  VT  (4) 

B  =  -70.47095  +  3.15543  •  KI  - 1.44857  •  CAPE  +  2.85664  •  CT  +  7.38063  •  LI  +  5.321  •  SWEAT  - 

1.13132  •  CAP  +  5.61748  •  VT  (5) 

A  =  -44.41068  -  0.0006132  ♦  CAPE  - 12.27451  •  LAPS75  +  0.42561  •  TT  +  0.13604  •  KI  (6) 

B  =  -46.00232  -  0.0003864  •  CAPE  - 12.5575  •  LAPS75  +  0.36563  •  TT  -  0.6864  •  KI  (7) 

A  =  -22.42869  +  0.8227 •  KI  +  1.80161  •  CAPE  -  0.63756  •  CT  +  4.88022  •  SWEAT  (8) 

A  =  -21.03699  +  0.31079  •  KI  +  2.01024  •  CAPE  -  0.34071  •  CT  +  4.65047  •  SWEAT  (9) 


4 


DISCRIMINATE  ANALYSIS 


percent  incorrect,  percent  missed,  and  false  alarm  rate 
for  each  of  the  classifications  for  the  “calibration”  data 


set.  A  similar  table  will  be  shown  in  Chapter  3  for  the 
verification  data  set. 


Table  2.  Classification  table  for  binned  Table  4.  Classification  table  for  binned 

unmodified  upper-air  data  modified  upper-air  data. 


Table  3.  Classification  table  for  unbinned  Table  5.  Classification  table  for  unbinned 

unmodified  upper-air  data.  modified  upper-air  data. 


Table  6.  Statistical  summary  for  the  calibration  data  showing  percent  correct,  percent  incorrect,  percent 
missed,  and  false  alarm  rate. 


MAXWELL  AFB  THUNDERSTORM  STUDY 

Chapter  3 

VERIFICATION 


3.1  Verification  Procedure.  The  discriminate 
functions  were  verified  against  an  independent  data 
set  consisting  of  upper-air  data  from  Centerville,  Ala., 
and  surface  data  from  both  Maxwell  AFB  and 
Montgomery,  Ala.  May  through  October  were  used 
over  the  POR 1975  through  1981 .  The  procedure  used 
was:  (1)  plug  in  the  values  of  the  appropriate  indices 
into  the  corresponding  coefficient  equations 
(Equations  2-9),  and  (2)  plug  those  coefficients  into 
the  discriminate  function  (Equation  1).  An  example 
calculation  for  a  thunderstorm  day  in  June  1977  for 
the  unmodified  binned  case  is  shown  below  (refer  to 
Equations  1 , 4,  and  5)  for  the  following  input  data:  KI 
=  32.3  or  bin  3,  CAPE  =  6093.6  or  bin  7,  CT  =  21.2  or 
bin  3,  LI  =  -5.9  or  bin  10,  SWEAT  =  184.8  or  bin  7, 
CAP  =  6.1  or  bin  7,  and  VT  =  27.5  or  bin  6.  Refer  to 
Table  1  for  the  binning  categories. 


A  =  -70.19581 +(3.6863 1 *3 .0)-(l  .7 549*7 .0)+ 
(2.49144*3.0)+(7.2467*10.0)+(5.54527*7.0)- 
(1 .08 803*7 .0)+(5 .4440 1*6.0)  =  72.38488 


B  =  -70.47095+(3.1 5543*3. 0)-(l. 44857*7.0)+ 
(2. 85  664*3 .0)+(7 .38063*1 0.0)+(5 .32 1  *7 .0)- 
(1.13132*7.0)+(5.61748*6.0)  =  74.26421 


P  =  l/[exp(72.38488-74.26421)+l]  =  0.868  or  86.8 
percent 


Table  7.  Classification  table  for  binned 


nodified  up] 

ner-air  data  (50 

percent). 

OBSERVED  1 

FORECAST 

NO 

YES 

Total 

NO 

616 

111 

727 

84.73 

15.27 

100.00 

YES 

250 

177 

427 

58.55  1 

41.45 

100.00 

Total 

866 

288 

1154 

Percent 

75.04 

24.96^ 

100.00 

Table  9.  Classification  table  for  binned 
modified  upper-air  data  (50  percent). 


OBSERVED  | 

FORECAST 

NO 

YES 

Total 

NO 

619 

114 

733 

84.45 

15.55 

100.00 

YES 

244 

174 

418 

58.37 

41.63 

100.00 

Total 

863 

288 

1151 

Percent 

74.98 

25.02 

100.00 

Contingency  tables  for  each  classification  were 
generated  using  Equations  1  through  9  for  each  day 
of  the  verification  POR  and  are  summarized  in  Tables 
7  through  30. 

3.2  Verification  Results.  Six  different  sets  of 
classification  tables  were  produced  from  the  output 
of  the  discriminate  function.  The  output  from  Equation 
1  is  a  number  between  0  and  1,  which  corresponds  to 
the  probability  between  0  and  100  percent  that 
thunderstorms  are  forecast  to  occur.  Six  probability 
thresholds  were  examined  in  order  to  look  at  the 
relationship  between  correct  forecasts,  incorrect 
forecasts,  false  alarm  rates,  and  probability  of 
detection.  These  probability  thresholds  were  used  to 
determine  whether  the  discriminant  function  forecast 
thunderstorms  to  occur  (“yes”)  or  not  (“no”).  For 
example,  in  the  first  set  of  tables  (Tables  7-10),  if  the 
probability  from  the  discriminate  equation  was  greater 
than  or  equal  to  0.5  or  50  percent  chance,  the  forecast 
was  a  “yes”  and  below  0.5  the  forecast  was  a  “no”. 
The  remaining  tables  (11-30)  use  higher  probability 
thresholds  incrementally  from  0.55  to  0.75  by  0.05 
(55  to  75  percent  every  5  percent).  The  data  from 
these  tables  can  be  used  to  calculate  measures  of  skill. 


Table  8.  Classification  table  for  unbinned 


nodified  up] 

per-air  data  (50  ] 

percent). 

OBSERVED 

FORECAST 

NO 

YES 

Total 

NO 

783 

217 

1000 

78.3 

21.7 

100.00 

YES 

83 

71 

154 

53.90 

46.1 

100.00 

Total 

866 

288 

1154 

Percent 

75.04 

24.96 

100.00 

Table  10.  Classification  table  for  unbinned 


OBSERVED 

FORECAST 

NO 

YES 

Total  j 

NO 

615 

106 

721 

85.30 

14.70 

100.00 

YES 

248 

182 

430 

57.67 

42.33 

100.00 

Total 

863 

288 

1151 

Percent 

74.98 

25.02 

100.00 
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Table  1 1 .  Classification  table  for  binned 
unmodified  upper-air  data  (55  percent). 


OBSE] 

R.VED 

FORECAST 

NO 

YES 

Total 

NO 

673 

137 

810 

83.09 

16.91 

100.00 

YES 

193 

151 

427 

56.10 

43.90 

100.00 

Total 

866 

288 

1154 

Percent 

75.04 

|  24.96 

100.00 

Table  1 3.  Classification  table  for  binned 
modified  upper-air  data  (55  percent). 


OBSEl 

IVED 

FORECAST 

NO 

YES 

Total 

NO 

664  ”| 

135 

799 

83.10 

16.90 

100.00 

YES 

199 

153 

352 

56.53  1 

43.47 

100.00 

Total 

863 

288 

1151 

Percent 

74.98 

25.02 

100.00 

Table  15.  Classification  table  for  binned 
unmodified  upper-air  data  (60  percent). 


OBSERVED 

FORECAST 

NO 

YES 

Total 

NO 

709 

160 

869 

81.59 

18.41 

100.00 

YES 

157 

128 

285 

55.09 

44.91 

100.00 

Total 

866 

288 

1154 

Percent 

75.04 

24.96 

100.00 

Table  12.  Classification  table  for  unbinned 
unmodified  upper-air  data  (55  percent). 


OBSERVED 

FORECAST 

NO 

YES 

Total 

NO 

807 

232 

1039 

77.7 

22.3 

100.00 

YES 

59 

56 

115 

51.3 

48.7 

100.00 

Total 

866 

288 

1154 

Percent 

75.04 

24.96 

100.00 

Table  14.  Classification  table  for  unbinned 
modified  upper-air  data  (55  percent). 


OBSEl 

IVED 

FORECAST 

NO 

YES 

Total 

NO 

667 

124 

791 

84.32 

15.68 

100.00 

YES 

196 

164 

360 

54.44 

45.56 

100.00 

Total 

863 

288 

1151 

Percent 

74.98 

25.02 

100.00 

Table  16.  Classification  table  for  unbinned 
unmodified  upper-air  data  (60  percent). 


- OBSERVED 

FORECAST" 

NO 

YES 

Total 

TTC5 - 

528 

235 

io55 

7777 

223 

ioo.oO 

"YES 

38 

30 

58  " 

33 3 

. '56.8 

ioo.oO 

"Total - 

555 

255 

1154 

Percent 

75.04 

24.96 

100.00 

Table  17.  Classification  table  for  binned 
modified  upper-air  data  (60  percent). 


Table  18.  Classification  table  for  unbinned 
modified  upper-air  data  (60  percent). 


OBSEl 

ilVED 

FORECAST 

NO  1 

YES 

Total 

NO  ~1 

705 

144 

849 

83.04 

16.96 

100.00 

YES 

158 

144 

302 

52.32 

47.68 

100.00 

Total 

863 

288 

1151 

Percent 

74.98 

25.02 

100.00 

OBSERVED 

FORECAST 

NO 

YES 

Total 

NO 

704 

157 

861 

81.77 

18.23 

100.00 

YES 

159 

131 

290 

54.83 

45.17 

100.00 

Total 

863 

288 

1151 

Percent 

74.98 

25.02 

100.00 
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Table  19.  Classification  table  for  binned 
unmodified  upper-air  data  (65  percent). 


Total 


Percent 


OBSERVED 

NO 

YES 

Total 

752 

193 

945 

79.58 

20.42 

100.00 

114 

92 

206 

55.34 

44.66 

100.00 

866 

288 

1154 

75.04 

24.96 

100.00 

Table  20.  Classification  table  for  unbinned 
unmodified  upper-air  data  (65  percent). 


Table  21.  Classification  table  for  binned 
modified  upper-air  data  (65  percent). 


FORECAST 


NO 


Total 


Percent 


OBSERVED 

NO 

YES 

756 

193 

79.66 

20.34 

107 

95 

52.97 

47.03 

863 

288 

74.98 

25.02 

Total 


949 


100.00 


202 


100.00 


Table  22.  Classification  table  for  unbinned 
modified  upper-air  data  (65  percent). 


FORECAST 


NO 


100.00 


Total 


Percent 


OBSERVED 

NO 

YES 

740 

184 

80.09 

19.91 

123 

104 

54.19~1 

45.81 

863 

288 

74.98 

25.02 

Total 


924 


100.00 


227 


100.00 


1151 


100.00 


Table  23.  Classification  table  for  binned 
unmodified  upper-air  data  (70  percent). 


Total 


1003 


100.00 


151 


100.00 


1154 


100.00 


Total 


OBSERVED 

NO 

YES 

786 

217 

78.36 

21.64 

80 

71 

52.98 

47.02 

866 

288 

1  75.04 

24.96 

Table  24.  Classification  table  for  unbinned 
unmodified  upper-air  data  (70  percent). 


Table  28.  Classification  table  for  binned  Tab,e  28-  Classification  table  for  unbmned 
modified  upper-air  data  (70  percent).  modified  upper-air  data  (70  percent). 


Chapter  3 


Table  27.  Classification  table  for  binned 
unmodified  upper-air  data  (75  percent). 


FORECAST 

NO 

YES 

Total 

NO 

812 

234 

1046 

77.63 

22.37 

100.00 

YES  ~~”"1 

54 

54 

108 

50.00 

50.00 

100.00 

Total 

866 

288 

1154 

Percent 

75.04 

24.96 

100.00 

Table  29.  Classification  table  for  binned 
modified  upper-air  data  (75  percent). 


— OBSERVED 

FORECAST 

NO 

YES 

Total 

NO 

827 

248 

1075 

76.93 

23.07 

100.00 

YES 

36 

40 

76 

47.37 

52.63 

100.00 

Total 

863 

288 

1151 

Percent 

74.98 

25.02 

100.00 

3.3  Skill  Scores.  Table  32  summarizes  skill  scores 
for  six  different  probability  thresholds  described  in 
section  3.2.  These  measures  of  skill  include  percent 
correct,  percent  incorrect,  probability  of  detection 
(POD),  percent  missed,  false  alarm  rate  (FAR),  the 
Heidke  Skill  Score  (HSS),  and  the  Hanssen  and 
Kuipers  Discriminate  “V”  (HKDV).  Equations  10 
through  16  are  used  to  calculate  measures  of  skill 
(using  the  parameters  A,  B,  C,  and  D  defined  in  Table 
31).  The  percent  correct  is  simply  the  sum  of  the  yes 
forecast/yes  observed  and  the  no  forecast/no  observed 
divided  by  the  total  number  of  forecast  made.  The 
percent  incorrect  is  one  minus  the  percent  correct.  The 
probability  of  detection  is  the  yes  forecast/yes 
observed  divided  by  the  sum  of  the  yes  forecast/yes 
observed  and  the  no  forecast/yes  observed.  The 
percent  missed  is  the  no  forecast/yes  observed  divided 
by  the  total  number  of  event  days  (no  forecast/yes 
observed + yes  forecast/yes  observed).  The  false  alarm 
rate  is  the  number  of  yes  forecast/no  observed  divided 


Table  28.  Classification  table  for  unbinned 
unmodified  upper- air  data  (75  percent). 


1 - i 

HHBH 

■ms 

BB 

HUB 

HUM 

TE5 

MM 

HUH 

IH2I 

warn 

mum 

Total 

msm 

Hia 

BUifiBi 

Percent 

mum 

msm 

Table  30.  Classification  table  for  binned 
unmodified  upper-air  data  (75  percent). 


OBSERVED 

FORECAST 

NO 

YES 

Total 

NO 

820 

248 

1068 

76.78 

23.22 

100.00 

YES 

43 

40 

83 

51.81 

48.19 

100.00 

Total 

863 

288 

1151 

Percent 

74.98 

25.02 

100.00 

by  the  sum  of  yes  forecast/yes  observed  and  yes 
forecast/no  observed.  The  Heidke  Skill  Score  and  the 
Hanssen  and  Kuipers  Discriminate  “V”  are  calculated 
using  Equation  15  and  16,  respectively.  The  Heidke 
Skill  Score  and  the  Hanssen  and  Kuipers  Discriminate 
“V”  assess  the  skill  of  the  forecast  method  to  that  of 
chance.  The  range  of  these  measures  is  from  -1  (no 
skill)  to  +1  (perfect  skill).  A  score  of  zero  implies  that 
the  method  is  no  better  than  chance. 


Table  31.  Contingency  Table. 


OBSERVED 

__ 

YES 

F 

Q 

R 

n 

•  #  of  *  ■■ 

-  f,  of  . 

E 

C 

Y 

A 

S 

.T 

E 

S 

#  of  occurrences  (C) 

#  of  occurrences  (D) 
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VERIFICATION 


The  ideal  situation  would  be  to  maximize  the  percent 
correct  and  the  probability  of  detection,  while 
minimizing  the  percent  missed  and  false  alarm  rate. 
From  this  table,  a  trend  is  evident:  as  the  probability 
threshold  increases  from  50  percent  to  75  percent,  the 
false  alarm  rate  decreases,  but  the  probability  of 
detection  decreases  and  the  percent  missed  increases. 
This  trend  is  understandable  since  by  using  a  higher 
probability  threshold,  the  yes  forecasts  that  are  made 

A+D 

Percent  Correct  = -  00) 

(A +5 + C+D) 

B+C 

Percent  Incorect  =  '■ - (11) 

(A+B+C+D) 

D 

Probability  of  Detection  - -  (12) 

(B+D) 


are  more  likely  to  verify  because  the  weather 
conditions  more  clearly  indicate  the  formation  of 
thunderstorms.  However,  this  causes  the 
thunderstorms  that  occur  in  the  lower  probability 
threshold  region  to  be  missed.  Thus,  the  probability 
of  detection  decreases  and  the  missed  forecasts 
increases.  It  is  left  to  the  user  to  define  an  acceptable 
balance  of  these  parameters. 

B 

Percent  Missed  - -  (13) 

(B+D) 

FAR  =  -  (14) 

(C+D) 

2(AD-BC) 

=  _ - — -  (15) 

(B+D)(B+A)+(C+D)(A+D) 

AD-BC 

HKDV  = -  (16) 

(B+D)(A+C) 


Table  32.  Summary  of  skill  table  showing  the  forecast  threshold,  percent  correct,  percent  incorrect,  probability 
of  detection,  percent  missed,  false  alarm  rate,  Heidke  skill  score,  and  Hanssen  and  Kuipers  Discriminate. 


index/sounding 

Threshold 

% 

Correct 

% 

Incorrect 

POD 

%  Missed 

■ 

HKDV 

unbinned/unmodified 

50% 

74.0 

26.0 

ESI 

75.3 

ESI! 

HI  i  d 

55% 

74.8 

25.2 

KESMI 

80.6 

E2&II 

B3 

_  1 

60% 

76.1 

23.9 

tail 

82.6 

f’HM 

0.130 

65% 

74.8 

25.2 

ESI 

91.0 

BO 

0.057 

70% 

75.2 

24.8 

EM 

93.4 

EZ9 

ga 

0.046 

75% 

75.0 

25.0 

EM 

96.2 

0.024 

unbinned/modified 

50% 

69.2 

30.8 

E-ttfl 

36.8 

57.7 

0.30  | 

0.345 

55% 

72.2 

27.8 

m» 

43.1 

0.342 

60% 

73.8 

2  62 

mi’ll 

50.0 

0.317 

65% 

73.3 

26.7 

EH 

63.9 

jgai 

0.219 

70% 

74.5 

25.5 

Kll 

76.6 

MM 

75% 

25.3 

HI 

86.1 

Esa 

binned/unmod  ified 

50% 

U£fl 

38.5 

KEM 

0.326 

55% 

71.4 

28.6 

m-rr* 

47.6 

0.301 

60% 

72.5 

27.5 

maa 

55.6 

lyfll 

0.263 

65% 

73.1 

26.9 

HO 

68.1 

El 

0.188 

70% 

74.3 

25.7 

IES1 

75.3 

0.154 

75% 

75.0 

25.0 

(18-8 

81.3 

iMflj 

0.125 

binned/modified 

50% 

68.9 

31.1 

wm 

39.6 

0.321 

55% 

71.0 

29.0 

ieh 

46.9 

iKfibl 

60% 

72.5 

27.5 

MUM 

54.5 

IE5I 

65% 

73.9 

26.1 

33.0 

67.0 

IESI 

70% 

75.1 

24.9 

24.3 

75.7 

IlgJ 

iaaa 

75% 

75.3 

|  24.7  13.9 

86.1 

igg 
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MAXWELL  AFB  THUNDERSTORM  STUDY 

Chapter  4 

SUMMARY,  CONCLUSIONS,  AND  RECOMMENDATIONS 


4.1  Summary.  Thunderstorm  prediction  continues 
to  be  an  important  and  challenging  aspect  of 
operational  aviation  forecasting  in  the  military.  Most 
of  research  and  investigation  has  been  devoted  to 
forecasting  the  intensity  of  thunderstorms — not 
whether  or  not  thunderstorms  will  occur  at  a  given 
location.  This  has  left  the  forecaster  with  the  tools  to 
forecast  how  severe  the  thunderstorm  would  be  if  it 
were  to  happen,  but  nothing  to  guide  them  on  how  to 
forecast  whether  or  not  thunderstorms  will  occur.  This 
study  was  an  attempt  to  provide  a  tool  to  this  end, 
using  standard  thunderstorm  indices.  Initial 
investigation  into  correlation  data  between  the 
occurrence/non-occurrence  of  thunderstorms  with 
eleven  thunderstorm  indices  showed  little  to  no 
correlation.  Therefore,  a  different  approach  was  used. 
Discriminate  analysis  was  used  to  develop  regression 
equations  relating  the  severe  thunderstorm  indices  to 
the  occurrence/non-occurrence  of  thunderstorms. 
These  equations  provide  the  forecaster  a  statistical 
probability  of  occurrence  of  thunderstorms. 

4.2  Conclusion.  The  regression  equations  developed 
to  provide  forecast  guidance  should  prove  to  be  useful 
if  they  are  used  as  a  tool — not  as  the  forecast.  As 
with  any  other  tool,  the  probability  output  from  the 
regression  equation  needs  to  be  melded  into  the 
forecasters  decision-making  process  to  make  the  best 


possible  forecast.  This  is  especially  true  given  the 
values  of  the  skill  scores.  The  combination  with  the 
best  skill  is  the  unbinned  modified  sounding  regression 
equation  using  the  55  percent  forecast  threshold. 
However,  the  probability  of  detection  decreases  and 
the  percent  missed  increases  as  you  move  away  from 
the  50  percent  forecast  threshold.  There  are  other 
very  important  factors  and  conditions  that  should  be 
included  in  the  thunderstorm  forecast,  such  as  the 
occurrence  of  thunderstorms  the  previous  day  and  the 
type  of  air  mass  that  exists. 

4.3  Recommendations.  Close  examination  of  Table 
32  shows  the  trade-offs  between  the  percent  correct 
forecasts  and  the  percent  of  missed  events  and  the 
false  alarm  rate.  Operational  forecasters  typically  err 
on  the  conservative  side.  That  is  to  say,  a  higher  false 
alarm  rate  is  usually  preferred  to  a  higher  percent 
missed  rate.  Therefore,  it  is  recommended  that  the 
50  percent  threshold  be  used  as  a  yes/no  breakpoint. 
This  will  minimize  the  percent  missed  and  the  false 
alarm  rate,  while  maximizing  the  probability  of 
detection.  Additionally,  from  the  skill  scores  in  Table 
32,  the  50  percent  threshold  shows  the  most  skill.  The 
best  combination  to  use  is  the  unbinned  modified 
sounding  regression  equation.  It  has  high  skill  scores, 
a  low  false  alarm  rate,  a  low  percent  missed,  and  a 
high  probability  of  detection. 
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GLOSSARY 


CAPE:  Convectively  Available  Potential  Energy  (CAPE)  is  an  index  that  measures  the  positive  area  on  the 
Skew-T  Log  P  diagram.  It  is  calculated  by  finding  the  lifted  condensation  level  (LCL)  and  lifting  that  parcel 
moist  adiabatically  through  the  level  of  free  convection  (LFC)  to  the  Equilibrium  Level.  The  area  between  this 
path  and  the  actual  environmental  temperature  trace  is  the  CAPE. 

Cap:  Cap  or  lid  strength  is  the  difference  between  buoyancy  strength  and  the  inversion  strength.  The  buoyancy 
strength  is  defined  as  the  difference  between  the  mean  wet-bulb  potential  temperature  in  the  lowest  50  millibars 
and  the  mean  saturation  wet-bulb  potential  temperature  between  the  inversion  and  the  500  millibar  level.  The 
inversion  strength  is  defined  as  the  difference  between  the  maximum  saturation  wet-bulb  potential  temperature 
at  the  inversion  level  and  the  mean  wet-bulb  potential  temperature  in  the  lowest  50  millibars. 

MEANRH:  MEANRH  is  the  mean  relative  humidity  between  the  surface  and  the  900  millibar  level. 

LAPS75:  LAPS75  is  the  lapse  rate  calculated  between  700  millibars  and  500  millibars. 

LI:  LI  is  an  acronym  that  stands  for  lifted  index.  The  lifted  index  is  defined  as  the  temperature  difference 
between  the  environment  and  a  parcel  of  air  that  has  been  lifted  moist  adiabatically  from  the  LCL  to  5 
millibars. 

VT:  VT  is  a  an  acronym  that  stands  for  vertical  totals.  It  is  calculated  by  finding  the  temperature  difference 
between  850  millibars  and  500  millibars. 

CT:  CT  is  a  an  acronym  that  stands  for  Cross  Totals.  It  is  calculated  by  finding  the  difference  between  the  dew 
point  temperature  at  850  millibars  and  the  ambient  temperature  at  500  millibars. 

TT:  IT  is  a  an  acronym  that  stands  for  Total  Totals.  It  is  calculated  by  summing  the  cross  totals  and  the  vertical 
totals. 

KI:  KI  is  a  an  acronym  that  stands  for  K-Index.  It  is  derived  mathematically  and  does  not  require  a  plotted 
sounding. 

SSI:  SSI  is  an  acronym  that  stands  for  Showalter  Stability  Index.  It  is  calculated  by  taking  the  difference 
between  the  ambient  temperature  at  500  millibars  and  the  parcel  temperature  lifted  moist  adiabatically  from  the 
850  millibar  LCL  to  500  millibars. 

SWEAT:  SWEAT  is  an  acronym  that  stands  for  Severe  Weather  Threat  index.  The  SWEAT  index  consists  of  a 
formula  that  uses  five  parameters  that  contribute  to  severe  weather  potential:  low-level  moisture  (850  millibar 
dew  point),  instability  (TT),  low-  and  upper-level  jet  (850  and  500  millibar  wind  speeds),  and  warm  advection 
(veering  wind  direction  between  850  and  500  millibar  levels),  to  calculate  a  score.  Commonly  accepted  SWEA1 
index  threshold  values  used  by  the  Air  Force  are  usually  300  for  severe  weather  and  400  for  tornadoes. 
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ACRONABs 


ACRONAB 

AFB 

AFCCC 

CAPE 

CT 

EL 

FAR 

GMT 

HKDY 

HSS 

KI 

LCL 

LFC 

LI 

POD 

POR 

SHARP 

SSI 

SWEAT 

TT 

USAFETAC 

VT 


Acronym  or  abbreviation 
Air  Force  Base 

Air  Force  Combat  Climatology  Center  (formerly  designated  USAFETAC) 

Convectively  Available  Potential  Energy 

Cross  Totals 

Equilibrium  Level 

False  Alarm  Rate 

Greenwich  Mean  Time 

Hanssen  and  Kuipers  Discriminant  V 

Heidke  Skill  Score 

K-Index 

Lifting  Condensation  Level 
Level  of  Free  Convection 
Lifted  Index 
Probability  Of  Detection 
Period  Of  Record 

Skew-T  Hodograph  Analysis  and  Research  Program 
Showalter  Stability  Index 
Severe  Weather  Threat 
Total  Totals 

United  States  Air  Force  Environmental  Technical  Applications  Center  (renamed 
AFCCC) 

Vertical  Totals 
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